Developer Guide

Procedure

1. Create a topic with the following request format:

   Syntax

   POST
   Action=CreateTopic
   &Name=TOPIC_NAME
   [&Attributes.entry.1.key=amqp-exchange&Attributes.entry.1.value=EXCHANGE]
   [&Attributes.entry.2.key=amqp-ack-level&Attributes.entry.2.value=none|broker|routable]
   [&Attributes.entry.3.key=verify-ssl&Attributes.entry.3.value=true|false]
   [&Attributes.entry.4.key=kafka-ack-level&Attributes.entry.4.value=none|broker]
   [&Attributes.entry.5.key=use-ssl&Attributes.entry.5.value=true|false]
   [&Attributes.entry.6.key=ca-location&Attributes.entry.6.value=FILE_PATH]
   [&Attributes.entry.7.key=OpaqueData&Attributes.entry.7.value=OPAQUE_DATA]
   [&Attributes.entry.8.key=push-endpoint&Attributes.entry.8.value=ENDPOINT]

   Here are the request parameters:

   • Endpoint: URL of an endpoint to send notifications to.
   • OpaqueData: opaque data is set in the topic configuration and added to all notifications triggered by the topic.

   • HTTP endpoint:

     • URL: http[s]://FQDN[: PORT ]
     • port defaults to: Use 80/443 for HTTP[S] accordingly.
     • verify-ssl: Indicates whether the server certificate is validated by the client or not. By default , it is true.

   • AMQP0.9.1 endpoint:

     • URL: amqp://[USER : PASSWORD @] FQDN [: PORT][/VHOST].
     • User and password defaults to: guest and guest respectively.
     • User and password can only be provided with HTTPS. Otherwise, the topic creation request is rejected.
     • port defaults to: 5672.
     • vhost defaults to: “/”
     • amqp-exchange: The exchanges must exist and be able to route messages based on topics. This is a mandatory parameter for AMQP0.9.1. Different topics pointing to the same endpoint must use the same exchange.

     • amqp-ack-level: No end to end acknowledgement is required, as messages may persist in the broker before being delivered into their final destination. Three acknowledgement methods exist:

       • none: Message is considered delivered if sent to the broker.
       • broker: By default the message is considered delivered if acknowledged by the broker.

       • routable: Message is considered delivered if the broker can route to a consumer.

         Note

         The key and value of a specific parameter does not have to reside in the same line, or in any specific order, but must use the same index. Attribute indexing does not need to be sequential or start from any specific value.

         Note

         The topic-name is used for the AMQP topic.

   • Kafka endpoint:

     • URL: kafka://[USER: PASSWORD @] FQDN[: PORT].
     • If use-ssl is set to false by default. If use-ssl is set to true, secure connection is used for connecting with the broker.
     • If ca-location is provided, and secure connection is used, the specified CA will be used, instead of the default one, to authenticate the broker.
     • User and password can only be provided over HTTP[S]. If not,topic creation request will be rejected.
     • User and password may only be provided together with use-ssl, if not, connection to the broker would fail.
     • port defaults to: 9092.

     • kafka-ack-level: no end to end acknowledgement required, as messages may persist in the broker before being delivered into their final destination. Two acknowledgement methods exist:

       • none: message is considered delivered if sent to the broker.
       • broker: By default, the message is considered delivered if acknowledged by the broker.

2. Create a response in the following format:

   Syntax

   <CreateTopicResponse xmlns="https://sns.amazonaws.com/doc/2010-03-31/">
       <CreateTopicResult>
           <TopicArn></TopicArn>
       </CreateTopicResult>
       <ResponseMetadata>
           <RequestId></RequestId>
       </ResponseMetadata>
   </CreateTopicResponse>

   Note

   The topic Amazon Resource Name (ARN) in the response will have the following format: arn:aws:sns:ZONE_GROUP:TENANT:TOPIC

   The following is an example of AMQP0.9.1 endpoint:

   Syntax

   "client.create_topic(Name='my-topic' , Attributes={'push-endpoint': 'amqp://127.0.0.1:5672', 'amqp-exchange': 'ex1', 'amqp-ack-level': 'broker'})"

Procedure

1. Get topic information with the following request format:

   Syntax

   POST
   Action=GetTopic
   &TopicArn=TOPIC_ARN

   Here is an example of the response format:

   <GetTopicResponse>
     <GetTopicRersult>
       <Topic>
         <User>
         </User>
         <Name>
         </Name>
         <EndPoint>
           <EndpointAddress>
           </EndpointAddress>
           <EndpointArgs>
           </EndpointArgs>
           <EndpointTopic>
           </EndpointTopic>
         </EndPoint>
         <TopicArn>
         </TopicArn>
         <OpaqueData>
         </OpaqueData>
       </Topic>
     </GetTopicResult>
     <ResponseMetadata>
       <RequestId>
       </RequestId>
     </ResponseMetadata>
   </GetTopicResponse>

   These are the tags and their definitions:

   • User: Name of the user that created the topic.
   • Name: Name of the topic.
   • EndpointAddress: The endpoint URL. If the endpoint URL contains user and password information, request must be made over HTTPS. If not, the topic get request will be rejected.
   • EndPointArgs: The endpoint arguments.
   • EndpointTopic: The topic name that will be sent to the endpoint, can be different than the above topic name.
   • TopicArn: Topic ARN.

Procedure

1. List topic information with the following request format:

   POST
   Action=ListTopics

   Here is an example of the response format:

   <ListTopicdResponse xmlns="https://sns.amazonaws.com/doc/2020-03-31/">
     <ListTopicsRersult>
       <Topics>
         <member>
           <User>
           </User>
           <Name>
           </Name>
           <EndPoint>
             <EndpointAddress>
             </EndpointAddress>
             <EndpointArgs>
             </EndpointArgs>
             <EndpointTopic>
             </EndpointTopic>
           </EndPoint>
           <TopicArn>
           </TopicArn>
           <OpaqueData>
           </OpaqueData>
         </member>
       </Topics>
     </ListTopicsResult>
     <ResponseMetadata>
       <RequestId>
       </RequestId>
     </ResponseMetadata>
   </ListTopicsResponse>

   Note

   If endpoint URL contains user and password information, in any of the topics, the request must be made over HTTPS. If not, topic list request is rejected.

Procedure

1. Delete a topic with the following request format:

   Syntax

   POST
   Action=DeleteTopic
   &TopicArn=TOPIC_ARN

   Here is an example of the response format:

   <DeleteTopicResponse xmlns="https://sns.amazonaws.com/doc/2020-03-31/">
     <ResponseMetadata>
       <RequestId>
       </RequestId>
     </ResponseMetadata>
   </DeleteTopicResponse>

1. Get the value of the header string.
2. Normalize the request header string into canonical form.
3. Generate an HMAC using a SHA-1 hashing algorithm.
4. Encode the hmac result as base-64.

1. Get all content- headers.
2. Remove all content- headers except for content-type and content-md5.
3. Ensure the content- header names are lowercase.
4. Sort the content- headers lexicographically.
5. Ensure you have a Date header AND ensure the specified date uses GMT and not an offset.
6. Get all headers beginning with x-amz-.
7. Ensure that the x-amz- headers are all lowercase.
8. Sort the x-amz- headers lexicographically.
9. Combine multiple instances of the same field name into a single field and separate the field values with a comma.
10. Replace white space and line breaks in header values with a single space.
11. Remove white space before and after colons.
12. Append a new line after each header.
13. Merge the headers back into the request header.

Procedure

1. As root, open port 8080 on firewall:

   [root@rgw ~]# firewall-cmd --zone=public --add-port=8080/tcp --permanent
   [root@rgw ~]# firewall-cmd --reload

2. Add a wildcard to the DNS server that you are using for the gateway as mentioned in the Object Gateway Configuration and Administration Guide.

   You can also set up the gateway node for local DNS caching. To do so, execute the following steps:

   1. As root, install and setup dnsmasq:

      [root@rgw ~]# yum install dnsmasq
      [root@rgw ~]# echo "address=/.FQDN_OF_GATEWAY_NODE/IP_OF_GATEWAY_NODE" | tee --append /etc/dnsmasq.conf
      [root@rgw ~]# systemctl start dnsmasq
      [root@rgw ~]# systemctl enable dnsmasq

      Replace IP_OF_GATEWAY_NODE and FQDN_OF_GATEWAY_NODE with the IP address and FQDN of the gateway node.

   2. As root, stop NetworkManager:

      [root@rgw ~]# systemctl stop NetworkManager
      [root@rgw ~]# systemctl disable NetworkManager

   3. As root, set the gateway server’s IP as the nameserver:

      [root@rgw ~]# echo "DNS1=IP_OF_GATEWAY_NODE" | tee --append /etc/sysconfig/network-scripts/ifcfg-eth0
      [root@rgw ~]# echo "IP_OF_GATEWAY_NODE FQDN_OF_GATEWAY_NODE" | tee --append /etc/hosts
      [root@rgw ~]# systemctl restart network
      [root@rgw ~]# systemctl enable network
      [root@rgw ~]# systemctl restart dnsmasq

      Replace IP_OF_GATEWAY_NODE and FQDN_OF_GATEWAY_NODE with the IP address and FQDN of the gateway node.

   4. Verify subdomain requests:

      [user@rgw ~]$ ping mybucket.FQDN_OF_GATEWAY_NODE

      Replace FQDN_OF_GATEWAY_NODE with the FQDN of the gateway node.

      Warning

      Setting up the gateway server for local DNS caching is for testing purposes only. You won’t be able to access outside network after doing this. It is strongly recommended to use a proper DNS server for the Red Hat Ceph Storage cluster and gateway node.

3. Create the radosgw user for S3 access carefully as mentioned in the Object Gateway Configuration and Administration Guide and copy the generated access_key and secret_key. You will need these keys for S3 access and subsequent bucket management tasks.

Procedure

1. Install the ruby package:

   [root@dev ~]# yum install ruby

   Note

   The above command will install ruby and it’s essential dependencies like rubygems and ruby-libs. If somehow the command does not install all the dependencies, install them separately.

2. Install the aws-s3 Ruby package:

   [root@dev ~]# gem install aws-s3

3. Create a project directory:

   [user@dev ~]$ mkdir ruby_aws_s3
   [user@dev ~]$ cd ruby_aws_s3

4. Create the connection file:

   [user@dev ~]$ vim conn.rb

5. Paste the following contents into the conn.rb file:

   Syntax

   #!/usr/bin/env ruby
   
   require 'aws/s3'
   require 'resolv-replace'
   
   AWS::S3::Base.establish_connection!(
           :server            => 'FQDN_OF_GATEWAY_NODE',
           :port           => '8080',
           :access_key_id     => 'MY_ACCESS_KEY',
           :secret_access_key => 'MY_SECRET_KEY'
   )

   Replace FQDN_OF_GATEWAY_NODE with the FQDN of the Ceph Object Gateway node. Replace MY_ACCESS_KEY and MY_SECRET_KEY with the access_key and secret_key that was generated when you created the radosgw user for S3 access as mentioned in the Red Hat Ceph Storage Object Gateway Configuration and Administration Guide.

   Example

   #!/usr/bin/env ruby
   
   require 'aws/s3'
   require 'resolv-replace'
   
   AWS::S3::Base.establish_connection!(
           :server            => 'testclient.englab.pnq.redhat.com',
           :port           => '8080',
           :access_key_id     => '98J4R9P22P5CDL65HKP8',
           :secret_access_key => '6C+jcaP0dp0+FZfrRNgyGA9EzRy25pURldwje049'
   )

   Save the file and exit the editor.

6. Make the file executable:

   [user@dev ~]$ chmod +x conn.rb

7. Run the file:

   [user@dev ~]$ ./conn.rb | echo $?

   If you have provided the values correctly in the file, the output of the command will be 0.

8. Create a new file for creating a bucket:

   [user@dev ~]$ vim create_bucket.rb

   Paste the following contents into the file:

   #!/usr/bin/env ruby
   
   load 'conn.rb'
   
   AWS::S3::Bucket.create('my-new-bucket1')

   Save the file and exit the editor.

9. Make the file executable:

   [user@dev ~]$ chmod +x create_bucket.rb

10. Run the file:

    [user@dev ~]$ ./create_bucket.rb

    If the output of the command is true it would mean that bucket my-new-bucket1 was created successfully.

11. Create a new file for listing owned buckets:

    [user@dev ~]$ vim list_owned_buckets.rb

    Paste the following content into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    AWS::S3::Service.buckets.each do |bucket|
            puts "{bucket.name}\t{bucket.creation_date}"
    end

    Save the file and exit the editor.

12. Make the file executable:

    [user@dev ~]$ chmod +x list_owned_buckets.rb

13. Run the file:

    [user@dev ~]$ ./list_owned_buckets.rb

    The output should look something like this:

    my-new-bucket1 2020-01-21 10:33:19 UTC

14. Create a new file for creating an object:

    [user@dev ~]$ vim create_object.rb

    Paste the following contents into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    AWS::S3::S3Object.store(
            'hello.txt',
            'Hello World!',
            'my-new-bucket1',
            :content_type => 'text/plain'
    )

    Save the file and exit the editor.

15. Make the file executable:

    [user@dev ~]$ chmod +x create_object.rb

16. Run the file:

    [user@dev ~]$ ./create_object.rb

    This will create a file hello.txt with the string Hello World!.

17. Create a new file for listing a bucket’s content:

    [user@dev ~]$ vim list_bucket_content.rb

    Paste the following content into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    new_bucket = AWS::S3::Bucket.find('my-new-bucket1')
    new_bucket.each do |object|
            puts "{object.key}\t{object.about['content-length']}\t{object.about['last-modified']}"
    end

    Save the file and exit the editor.

18. Make the file executable.

    [user@dev ~]$ chmod +x list_bucket_content.rb

19. Run the file:

    [user@dev ~]$ ./list_bucket_content.rb

    The output will look something like this:

    hello.txt    12    Fri, 22 Jan 2020 15:54:52 GMT

20. Create a new file for deleting an empty bucket:

    [user@dev ~]$ vim del_empty_bucket.rb

    Paste the following contents into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    AWS::S3::Bucket.delete('my-new-bucket1')

    Save the file and exit the editor.

21. Make the file executable:

    [user@dev ~]$ chmod +x del_empty_bucket.rb

22. Run the file:

    [user@dev ~]$ ./del_empty_bucket.rb | echo $?

    If the bucket is successfully deleted, the command will return 0 as output.

    Note

    Edit the create_bucket.rb file to create empty buckets, for example: my-new-bucket4, my-new-bucket5. Next, edit the above mentioned del_empty_bucket.rb file accordingly before trying to delete empty buckets.

23. Create a new file for deleting non-empty buckets:

    [user@dev ~]$ vim del_non_empty_bucket.rb

    Paste the following contents into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    AWS::S3::Bucket.delete('my-new-bucket1', :force => true)

    Save the file and exit the editor.

24. Make the file executable:

    [user@dev ~]$ chmod +x del_non_empty_bucket.rb

25. Run the file:

    [user@dev ~]$ ./del_non_empty_bucket.rb | echo $?

    If the bucket is successfully deleted, the command will return 0 as output.

26. Create a new file for deleting an object:

    [user@dev ~]$ vim delete_object.rb

    Paste the following contents into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    AWS::S3::S3Object.delete('hello.txt', 'my-new-bucket1')

    Save the file and exit the editor.

27. Make the file executable:

    [user@dev ~]$ chmod +x delete_object.rb

28. Run the file:

    [user@dev ~]$ ./delete_object.rb

    This will delete the object hello.txt.

Procedure

1. Install the ruby package:

   [root@dev ~]# yum install ruby

   Note

   The above command will install ruby and it’s essential dependencies like rubygems and ruby-libs. If somehow the command does not install all the dependencies, install them separately.

2. Install the aws-sdk Ruby package:

   [root@dev ~]# gem install aws-sdk

3. Create a project directory:

   [user@dev ~]$ mkdir ruby_aws_sdk
   [user@dev ~]$ cd ruby_aws_sdk

4. Create the connection file:

   [user@ruby_aws_sdk]$ vim conn.rb

5. Paste the following contents into the conn.rb file:

   Syntax

   #!/usr/bin/env ruby
   
   require 'aws-sdk'
   require 'resolv-replace'
   
   Aws.config.update(
           endpoint: 'http://FQDN_OF_GATEWAY_NODE:8080',
           access_key_id: 'MY_ACCESS_KEY',
           secret_access_key: 'MY_SECRET_KEY',
           force_path_style: true,
           region: 'us-east-1'
   )

   Replace FQDN_OF_GATEWAY_NODE with the FQDN of the Ceph Object Gateway node. Replace MY_ACCESS_KEY and MY_SECRET_KEY with the access_key and secret_key that was generated when you created the radosgw user for S3 access as mentioned in the Red Hat Ceph Storage Object Gateway Configuration and Administration Guide.

   Example

   #!/usr/bin/env ruby
   
   require 'aws-sdk'
   require 'resolv-replace'
   
   Aws.config.update(
           endpoint: 'http://testclient.englab.pnq.redhat.com:8080',
           access_key_id: '98J4R9P22P5CDL65HKP8',
           secret_access_key: '6C+jcaP0dp0+FZfrRNgyGA9EzRy25pURldwje049',
           force_path_style: true,
           region: 'us-east-1'
   )

   Save the file and exit the editor.

6. Make the file executable:

   [user@ruby_aws_sdk]$ chmod +x conn.rb

7. Run the file:

   [user@ruby_aws_sdk]$ ./conn.rb | echo $?

   If you have provided the values correctly in the file, the output of the command will be 0.

8. Create a new file for creating a bucket:

   [user@ruby_aws_sdk]$ vim create_bucket.rb

   Paste the following contents into the file:

   Syntax

   #!/usr/bin/env ruby
   
   load 'conn.rb'
   
   s3_client = Aws::S3::Client.new
   s3_client.create_bucket(bucket: 'my-new-bucket2')

   Save the file and exit the editor.

9. Make the file executable:

   [user@ruby_aws_sdk]$ chmod +x create_bucket.rb

10. Run the file:

    [user@ruby_aws_sdk]$ ./create_bucket.rb

    If the output of the command is true, this means that bucket my-new-bucket2 was created successfully.

11. Create a new file for listing owned buckets:

    [user@ruby_aws_sdk]$ vim list_owned_buckets.rb

    Paste the following content into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    s3_client = Aws::S3::Client.new
    s3_client.list_buckets.buckets.each do |bucket|
            puts "{bucket.name}\t{bucket.creation_date}"
    end

    Save the file and exit the editor.

12. Make the file executable:

    [user@ruby_aws_sdk]$ chmod +x list_owned_buckets.rb

13. Run the file:

    [user@ruby_aws_sdk]$ ./list_owned_buckets.rb

    The output should look something like this:

    my-new-bucket2 2022-04-21 10:33:19 UTC

14. Create a new file for creating an object:

    [user@ruby_aws_sdk]$ vim create_object.rb

    Paste the following contents into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    s3_client = Aws::S3::Client.new
    s3_client.put_object(
            key: 'hello.txt',
            body: 'Hello World!',
            bucket: 'my-new-bucket2',
            content_type: 'text/plain'
    )

    Save the file and exit the editor.

15. Make the file executable:

    [user@ruby_aws_sdk]$ chmod +x create_object.rb

16. Run the file:

    [user@ruby_aws_sdk]$ ./create_object.rb

    This will create a file hello.txt with the string Hello World!.

17. Create a new file for listing a bucket’s content:

    [user@ruby_aws_sdk]$ vim list_bucket_content.rb

    Paste the following content into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    s3_client = Aws::S3::Client.new
    s3_client.list_objects(bucket: 'my-new-bucket2').contents.each do |object|
            puts "{object.key}\t{object.size}"
    end

    Save the file and exit the editor.

18. Make the file executable.

    [user@ruby_aws_sdk]$ chmod +x list_bucket_content.rb

19. Run the file:

    [user@ruby_aws_sdk]$ ./list_bucket_content.rb

    The output will look something like this:

    hello.txt    12    Fri, 22 Apr 2022 15:54:52 GMT

20. Create a new file for deleting an empty bucket:

    [user@ruby_aws_sdk]$ vim del_empty_bucket.rb

    Paste the following contents into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    s3_client = Aws::S3::Client.new
    s3_client.delete_bucket(bucket: 'my-new-bucket2')

    Save the file and exit the editor.

21. Make the file executable:

    [user@ruby_aws_sdk]$ chmod +x del_empty_bucket.rb

22. Run the file:

    [user@ruby_aws_sdk]$ ./del_empty_bucket.rb | echo $?

    If the bucket is successfully deleted, the command will return 0 as output.

    Note

    Edit the create_bucket.rb file to create empty buckets, for example: my-new-bucket6, my-new-bucket7. Next, edit the above mentioned del_empty_bucket.rb file accordingly before trying to delete empty buckets.

23. Create a new file for deleting a non-empty bucket:

    [user@ruby_aws_sdk]$ vim del_non_empty_bucket.rb

    Paste the following contents into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    s3_client = Aws::S3::Client.new
    Aws::S3::Bucket.new('my-new-bucket2', client: s3_client).clear!
    s3_client.delete_bucket(bucket: 'my-new-bucket2')

    Save the file and exit the editor.

24. Make the file executable:

    [user@ruby_aws_sdk]$ chmod +x del_non_empty_bucket.rb

25. Run the file:

    [user@ruby_aws_sdk]$ ./del_non_empty_bucket.rb | echo $?

    If the bucket is successfully deleted, the command will return 0 as output.

26. Create a new file for deleting an object:

    [user@ruby_aws_sdk]$ vim delete_object.rb

    Paste the following contents into the file:

    #!/usr/bin/env ruby
    
    load 'conn.rb'
    
    s3_client = Aws::S3::Client.new
    s3_client.delete_object(key: 'hello.txt', bucket: 'my-new-bucket2')

    Save the file and exit the editor.

27. Make the file executable:

    [user@ruby_aws_sdk]$ chmod +x delete_object.rb

28. Run the file:

    [user@ruby_aws_sdk]$ ./delete_object.rb

    This will delete the object hello.txt.

Procedure

1. Install the php package:

   [root@dev ~]# yum install php

2. Download the zip archive of aws-sdk for PHP and extract it.

3. Create a project directory:

   [user@dev ~]$ mkdir php_s3
   [user@dev ~]$ cd php_s3

4. Copy the extracted aws directory to the project directory. For example:

   [user@php_s3]$ cp -r ~/Downloads/aws/ ~/php_s3/

5. Create the connection file:

   [user@php_s3]$ vim conn.php

6. Paste the following contents in the conn.php file:

   Syntax

   <?php
   define('AWS_KEY', 'MY_ACCESS_KEY');
   define('AWS_SECRET_KEY', 'MY_SECRET_KEY');
   define('HOST', 'FQDN_OF_GATEWAY_NODE');
   define('PORT', '8080');
   
   // require the AWS SDK for php library
   require '/PATH_TO_AWS/aws-autoloader.php';
   
   use Aws\S3\S3Client;
   
   // Establish connection with host using S3 Client
   client = S3Client::factory(array(
       'base_url' => HOST,
       'port' => PORT,
       'key'      => AWS_KEY,
       'secret'   => AWS_SECRET_KEY
   ));
   ?>

   Replace FQDN_OF_GATEWAY_NODE with the FQDN of the gateway node. Replace MY_ACCESS_KEY and MY_SECRET_KEY with the access_key and secret_key that was generated when creating the radosgw user for S3 access as mentioned in the Red Hat Ceph Storage Object Gateway Configuration and Administration Guide. Replace PATH_TO_AWS with the absolute path to the extracted aws directory that you copied to the php project directory.

   Save the file and exit the editor.

7. Run the file:

   [user@php_s3]$ php -f conn.php | echo $?

   If you have provided the values correctly in the file, the output of the command will be 0.

8. Create a new file for creating a bucket:

   [user@php_s3]$ vim create_bucket.php

   Paste the following contents into the new file:

   Syntax

   <?php
   
   include 'conn.php';
   
   client->createBucket(array('Bucket' => 'my-new-bucket3'));
   
   ?>

   Save the file and exit the editor.

9. Run the file:

   [user@php_s3]$ php -f create_bucket.php

10. Create a new file for listing owned buckets:

    [user@php_s3]$ vim list_owned_buckets.php

    Paste the following content into the file:

    Syntax

    <?php
    
    include 'conn.php';
    
    blist = client->listBuckets();
    echo "Buckets belonging to " . blist['Owner']['ID'] . ":\n";
    foreach (blist['Buckets'] as b) {
        echo "{b['Name']}\t{b['CreationDate']}\n";
    }
    
    ?>

    Save the file and exit the editor.

11. Run the file:

    [user@php_s3]$ php -f list_owned_buckets.php

    The output should look similar to this:

    my-new-bucket3 2022-04-21 10:33:19 UTC

12. Create an object by first creating a source file named hello.txt:

    [user@php_s3]$ echo "Hello World!" > hello.txt

13. Create a new php file:

    [user@php_s3]$ vim create_object.php

    Paste the following contents into the file:

    Syntax

    <?php
    
    include 'conn.php';
    
    key         = 'hello.txt';
    source_file = './hello.txt';
    acl         = 'private';
    bucket      = 'my-new-bucket3';
    client->upload(bucket, key, fopen(source_file, 'r'), acl);
    
    ?>

    Save the file and exit the editor.

14. Run the file:

    [user@php_s3]$ php -f create_object.php

    This will create the object hello.txt in bucket my-new-bucket3.

15. Create a new file for listing a bucket’s content:

    [user@php_s3]$ vim list_bucket_content.php

    Paste the following content into the file:

    Syntax

    <?php
    
    include 'conn.php';
    
    o_iter = client->getIterator('ListObjects', array(
        'Bucket' => 'my-new-bucket3'
    ));
    foreach (o_iter as o) {
        echo "{o['Key']}\t{o['Size']}\t{o['LastModified']}\n";
    }
    ?>

    Save the file and exit the editor.

16. Run the file:

    [user@php_s3]$ php -f list_bucket_content.php

    The output will look similar to this:

    hello.txt    12    Fri, 22 Apr 2022 15:54:52 GMT

17. Create a new file for deleting an empty bucket:

    [user@php_s3]$ vim del_empty_bucket.php

    Paste the following contents into the file:

    Syntax

    <?php
    
    include 'conn.php';
    
    client->deleteBucket(array('Bucket' => 'my-new-bucket3'));
    ?>

    Save the file and exit the editor.

18. Run the file:

    [user@php_s3]$ php -f del_empty_bucket.php | echo $?

    If the bucket is successfully deleted, the command will return 0 as output.

    Note

    Edit the create_bucket.php file to create empty buckets, for example: my-new-bucket4, my-new-bucket5. Next, edit the above mentioned del_empty_bucket.php file accordingly before trying to delete empty buckets.

    Important

    Deleting a non-empty bucket is currently not supported in PHP 2 and newer versions of aws-sdk.

19. Create a new file for deleting an object:

    [user@php_s3]$ vim delete_object.php

    Paste the following contents into the file:

    Syntax

    <?php
    
    include 'conn.php';
    
    client->deleteObject(array(
        'Bucket' => 'my-new-bucket3',
        'Key'    => 'hello.txt',
    ));
    ?>

    Save the file and exit the editor.

20. Run the file:

    [user@php_s3]$ php -f delete_object.php

    This will delete the object hello.txt.

Procedure

1. Install the awscli package:

   [user@dev]$ pip3 install --user awscli

2. Configure awscli to access Ceph Object Storage using AWS CLI:

   Syntax

   aws configure --profile=MY_PROFILE_NAME
   
   AWS Access Key ID [None]: MY_ACCESS_KEY
   AWS Secret Access Key [None]: MY_SECRET_KEY
   Default region name [None]:
   Default output format [None]:

   Replace MY_PROFILE_NAME with the name you want to use to identify this profile. Replace MY_ACCESS_KEY and MY_SECRET_KEY with the access_key and secret_key that was generated when creating the radosgw user for S3 access as mentioned in the Red Hat Ceph Storage Object Gateway Configuration and Administration Guide.

   Example

   [user@dev]$ aws configure --profile=ceph
   
   AWS Access Key ID [None]: 12345
   AWS Secret Access Key [None]: 67890
   Default region name [None]:
   Default output format [None]:

3. Create an alias to point to the FQDN of your Ceph Object Gateway node:

   Syntax

   alias aws="aws --endpoint-url=http://FQDN_OF_GATEWAY_NODE:8080"

   Replace FQDN_OF_GATEWAY_NODE with the FQDN of the Ceph Object Gateway node.

   Example

   [user@dev]$ alias aws="aws --endpoint-url=http://testclient.englab.pnq.redhat.com:8080"

4. Create a new bucket:

   Syntax

   aws --profile=MY_PROFILE_NAME s3api create-bucket --bucket BUCKET_NAME

   Replace MY_PROFILE_NAME with the name you created to use this profile. Replace BUCKET_NAME with a name for your new bucket.

   Example

   [user@dev]$ aws --profile=ceph s3api create-bucket --bucket mybucket

5. List owned buckets:

   Syntax

   aws --profile=MY_PROFILE_NAME s3api list-buckets

   Replace MY_PROFILE_NAME with the name you created to use this profile.

   Example

   [user@dev]$ aws --profile=ceph s3api list-buckets
   {
       "Buckets": [
           {
               "Name": "mybucket",
               "CreationDate": "2021-08-31T16:46:15.257Z"
           }
       ],
       "Owner": {
           "DisplayName": "User",
           "ID": "user"
       }
   }

6. Configure a bucket for MFA-Delete:

   Syntax

   aws --profile=MY_PROFILE_NAME s3api put-bucket-versioning --bucket BUCKET_NAME --versioning-configuration '{"Status":"Enabled","MFADelete":"Enabled"}' --mfa 'TOTP_SERIAL TOTP_PIN'

   • Replace MY_PROFILE_NAME with the name you created to use this profile.
   • Replace BUCKET_NAME with the name of your new bucket.
   • Replace TOTP_SERIAL with the string the represents the ID for the TOTP token and replace TOTP_PIN with the current pin displayed on your MFA authentication device.
   • The TOTP_SERIAL is the string that was specified when you created the radosgw user for S3.
   • See the Creating a new multi-factor authentication TOTP token section of the Red Hat Ceph Storage Object Gateway Configuration and Administration Guide for more details on creating a MFA TOTP token.

   • See the Creating a seed for multi-factor authentication using oathtool section in the Red Hat Ceph Storage Developer Guide for details on creating a MFA seed with oathtool.

     Example

     [user@dev]$ aws --profile=ceph s3api put-bucket-versioning --bucket mybucket --versioning-configuration '{"Status":"Enabled","MFADelete":"Enabled"}' --mfa 'MFAtest 232009'

7. View MFA-Delete status of the bucket versioning state:

   Syntax

   aws --profile=MY_PROFILE_NAME s3api get-bucket-versioning --bucket BUCKET_NAME

   Replace MY_PROFILE_NAME with the name you created to use this profile. Replace BUCKET_NAME with the name of your new bucket.

   Example

   [user@dev]$ aws --profile=ceph s3api get-bucket-versioning --bucket mybucket
   {
       "Status": "Enabled",
       "MFADelete": "Enabled"
   }

8. Add an object to the MFA-Delete enabled bucket:

   Syntax

   aws --profile=MY_PROFILE_NAME s3api put-object --bucket BUCKET_NAME --key OBJECT_KEY --body LOCAL_FILE

   • Replace MY_PROFILE_NAME with the name you created to use this profile.
   • Replace BUCKET_NAME with the name of your new bucket.
   • Replace OBJECT_KEY with the name that will uniquely identify the object in a bucket.

   • Replace LOCAL_FILE with the name of the local file to upload.

     Example

     [user@dev]$ aws --profile=ceph s3api put-object --bucket mybucket --key example --body testfile
     {
         "ETag": "\"5679b828547a4b44cfb24a23fd9bb9d5\"",
         "VersionId": "3VyyYPTEuIofdvMPWbr1znlOu7lJE3r"
     }

9. List the object versions for a specific object:

   Syntax

   aws --profile=MY_PROFILE_NAME s3api list-object-versions --bucket BUCKET_NAME --key OBJEC_KEY]

   • Replace MY_PROFILE_NAME with the name you created to use this profile.
   • Replace BUCKET_NAME with the name of your new bucket.

   • Replace OBJECT_KEY with the name that was specified to uniquely identify the object in a bucket.

     Example

     [user@dev]$ aws --profile=ceph s3api list-object-versions --bucket mybucket --key example
     {
         "IsTruncated": false,
         "KeyMarker": "example",
         "VersionIdMarker": "",
         "Versions": [
             {
                 "ETag": "\"5679b828547a4b44cfb24a23fd9bb9d5\"",
                 "Size": 196,
                 "StorageClass": "STANDARD",
                 "Key": "example",
                 "VersionId": "3VyyYPTEuIofdvMPWbr1znlOu7lJE3r",
                 "IsLatest": true,
                 "LastModified": "2021-08-31T17:48:45.484Z",
                 "Owner": {
                     "DisplayName": "User",
                     "ID": "user"
                 }
             }
         ],
         "Name": "mybucket",
         "Prefix": "",
         "MaxKeys": 1000,
         "EncodingType": "url"
     }

10. Delete an object in an MFA-Delete enabled bucket:

    Syntax

    aws --profile=MY_PROFILE_NAME s3api delete-object --bucket BUCKET_NAME --key OBJECT_KEY --version-id VERSION_ID --mfa 'TOTP_SERIAL TOTP_PIN'

    • Replace MY_PROFILE_NAME with the name you created to use this profile.
    • Replace BUCKET_NAME with the name of your bucket that contains the object to delete.
    • Replace OBJECT_KEY with the name that uniquely identifies the object in a bucket.
    • Replace VERSION_ID with the VersionID of the specific version of the object you want to delete.

    • Replace TOTP_SERIAL with the string that represents the ID for the TOTP token and TOTP_PIN to the current pin displayed on your MFA authentication device.

      Example

      [user@dev]$ aws --profile=ceph s3api delete-object --bucket mybucket --key example --version-id 3VyyYPTEuIofdvMPWbr1znlOu7lJE3r --mfa 'MFAtest 420797'
      {
          "VersionId": "3VyyYPTEuIofdvMPWbr1znlOu7lJE3r"
      }

      If the MFA token is not included, the request fails with the error shown below.

      Example

      [user@dev]$ aws --profile=ceph s3api delete-object --bucket mybucket --key example --version-id 3VyyYPTEuIofdvMPWbr1znlOu7lJE3r
      An error occurred (AccessDenied) when calling the DeleteObject operation: Unknown

11. List object versions to verify object was deleted from MFA-Delete enabled bucket:

    Syntax

    aws --profile=MY_PROFILE_NAME s3api list-object-versions --bucket BUCKET_NAME --key OBJECT_KEY

    • Replace MY_PROFILE_NAME with the name you created to use this profile.
    • Replace BUCKET_NAME with the name of your bucket.

    • Replace OBJECT_KEY with the name that uniquely identifies the object in a bucket.

      Example

      [user@dev]$ aws --profile=ceph s3api list-object-versions --bucket mybucket --key example
      {
          "IsTruncated": false,
          "KeyMarker": "example",
          "VersionIdMarker": "",
          "Name": "mybucket",
          "Prefix": "",
          "MaxKeys": 1000,
          "EncodingType": "url"
      }

Procedure

1. Install the oathtool package:

   [root@dev]# dnf install oathtool

2. Install the qrencode package:

   [root@dev]# dnf install qrencode

3. Generate a 30 character seed from the urandom Linux device file and store it in the shell variable SEED:

   Example

   [user@dev]$ SEED=$(head -10 /dev/urandom | sha512sum | cut -b 1-30)

4. Print the seed by running echo on the SEED variable:

   Example

   [user@dev]$ echo $SEED
   BA6GLJBJIKC3D7W7YFYXXAQ7

5. Feed the SEED into the oathtool command:

   Syntax

   oathtool -v -d6 $SEED

   Example

   [user@dev]$ oathtool -v -d6 $SEED
   Hex secret: 083c65a4294285b1fedfc1717b821f
   Base32 secret: BA6GLJBJIKC3D7W7YFYXXAQ7
   Digits: 6
   Window size: 0
   Start counter: 0x0 (0)
   
   823182

   Note

   The base32 secret is needed to add a token to the authenticator application on your MFA device. You can either use the QR code to import the token into the authenticator application or use the base32 secret manually add it.

6. Optional: Create a QR code image file to add the token to the authenticator:

   Syntax

   qrencode -o /tmp/user.png 'otpauth://totp/TOTP_SERIAL?secret=_BASE32_SECRET'

   Replace TOTP_SERIAL with the string that represents the ID for the (TOTP) token and BASE32_SECRET with the Base32 secret generated by oathtool.

   Example

   [user@dev]$ qrencode -o /tmp/user.png 'otpauth://totp/MFAtest?secret=BA6GLJBJIKC3D7W7YFYXXAQ7'

7. Scan the generated QR code image file to add the token to the authenticator application on your MFA device.
8. Create the multi-factor authentication TOTP token for the user using the radowgw-admin command.